{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 6-6 使用tensorflow-serving部署模型\n",
    "\n",
    "TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。\n",
    "\n",
    "例如：通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。\n",
    "\n",
    "通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。\n",
    "\n",
    "通过 tensorflow-serving 可以加载模型后提供网络接口API服务，通过任意编程语言发送网络请求都可以获取模型预测结果。\n",
    "\n",
    "通过 tensorFlow for Java接口，可以在Java或者spark(scala)中调用tensorflow模型进行预测。\n",
    "\n",
    "我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 〇、tensorflow serving模型部署概述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用 tensorflow serving 部署模型要完成以下步骤。\n",
    "\n",
    "* (1) 准备protobuf模型文件。\n",
    "\n",
    "* (2) 安装tensorflow serving。\n",
    "\n",
    "* (3) 启动tensorflow serving 服务。\n",
    "\n",
    "* (4) 向API服务发送请求，获取预测结果。\n",
    "\n",
    "\n",
    "可通过以下colab链接测试效果《tf_serving》：\n",
    "https://colab.research.google.com/drive/1vS5LAYJTEn-H0GDb1irzIuyRB8E3eWc8\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "inputs (InputLayer)          [(None, 2)]               0         \n",
      "_________________________________________________________________\n",
      "outputs (Dense)              (None, 1)                 3         \n",
      "=================================================================\n",
      "Total params: 3\n",
      "Trainable params: 3\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 800 samples\n",
      "Epoch 1/100\n",
      "800/800 [==============================] - 0s 281us/sample - loss: 337.4876 - mae: 15.6857\n",
      "Epoch 2/100\n",
      "800/800 [==============================] - 0s 82us/sample - loss: 314.9609 - mae: 15.1681\n",
      "Epoch 3/100\n",
      "800/800 [==============================] - 0s 90us/sample - loss: 293.6427 - mae: 14.6632\n",
      "Epoch 4/100\n",
      "800/800 [==============================] - 0s 86us/sample - loss: 273.3989 - mae: 14.1629\n",
      "Epoch 5/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 254.0911 - mae: 13.6659\n",
      "Epoch 6/100\n",
      "800/800 [==============================] - 0s 79us/sample - loss: 234.8635 - mae: 13.1567\n",
      "Epoch 7/100\n",
      "800/800 [==============================] - 0s 83us/sample - loss: 217.1905 - mae: 12.6616\n",
      "Epoch 8/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 200.6012 - mae: 12.1734\n",
      "Epoch 9/100\n",
      "800/800 [==============================] - 0s 79us/sample - loss: 184.5093 - mae: 11.6816\n",
      "Epoch 10/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 169.3151 - mae: 11.1943\n",
      "Epoch 11/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 154.9977 - mae: 10.7069\n",
      "Epoch 12/100\n",
      "800/800 [==============================] - 0s 81us/sample - loss: 141.3833 - mae: 10.2221\n",
      "Epoch 13/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 128.5914 - mae: 9.7499\n",
      "Epoch 14/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 116.2733 - mae: 9.2711\n",
      "Epoch 15/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 104.6660 - mae: 8.7932\n",
      "Epoch 16/100\n",
      "800/800 [==============================] - 0s 81us/sample - loss: 93.7081 - mae: 8.3192\n",
      "Epoch 17/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 83.6246 - mae: 7.8498\n",
      "Epoch 18/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 74.1360 - mae: 7.3879\n",
      "Epoch 19/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 65.4168 - mae: 6.9274\n",
      "Epoch 20/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 57.3533 - mae: 6.4729\n",
      "Epoch 21/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 49.7182 - mae: 6.0190\n",
      "Epoch 22/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 42.8253 - mae: 5.5718\n",
      "Epoch 23/100\n",
      "800/800 [==============================] - 0s 79us/sample - loss: 36.4617 - mae: 5.1218\n",
      "Epoch 24/100\n",
      "800/800 [==============================] - ETA: 0s - loss: 30.8805 - mae: 4.69 - 0s 78us/sample - loss: 30.7143 - mae: 4.6803\n",
      "Epoch 25/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 25.5962 - mae: 4.2427\n",
      "Epoch 26/100\n",
      "800/800 [==============================] - 0s 83us/sample - loss: 21.0802 - mae: 3.8293\n",
      "Epoch 27/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 17.1093 - mae: 3.4183\n",
      "Epoch 28/100\n",
      "800/800 [==============================] - 0s 79us/sample - loss: 13.8251 - mae: 3.0379\n",
      "Epoch 29/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 11.0200 - mae: 2.6861\n",
      "Epoch 30/100\n",
      "800/800 [==============================] - 0s 86us/sample - loss: 8.8733 - mae: 2.3937\n",
      "Epoch 31/100\n",
      "800/800 [==============================] - 0s 91us/sample - loss: 7.1994 - mae: 2.1475\n",
      "Epoch 32/100\n",
      "800/800 [==============================] - 0s 89us/sample - loss: 6.1093 - mae: 1.9852\n",
      "Epoch 33/100\n",
      "800/800 [==============================] - 0s 81us/sample - loss: 5.4566 - mae: 1.8780\n",
      "Epoch 34/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 5.0935 - mae: 1.8113\n",
      "Epoch 35/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 4.8766 - mae: 1.7689\n",
      "Epoch 36/100\n",
      "800/800 [==============================] - 0s 72us/sample - loss: 4.7155 - mae: 1.7373\n",
      "Epoch 37/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 4.5751 - mae: 1.7116\n",
      "Epoch 38/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 4.4554 - mae: 1.6905\n",
      "Epoch 39/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 4.3603 - mae: 1.6744\n",
      "Epoch 40/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 4.2710 - mae: 1.6603\n",
      "Epoch 41/100\n",
      "800/800 [==============================] - 0s 82us/sample - loss: 4.1952 - mae: 1.6466\n",
      "Epoch 42/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 4.1276 - mae: 1.6364\n",
      "Epoch 43/100\n",
      "800/800 [==============================] - 0s 88us/sample - loss: 4.0752 - mae: 1.6278\n",
      "Epoch 44/100\n",
      "800/800 [==============================] - 0s 83us/sample - loss: 4.0358 - mae: 1.6214\n",
      "Epoch 45/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 4.0002 - mae: 1.6151\n",
      "Epoch 46/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 3.9702 - mae: 1.6088\n",
      "Epoch 47/100\n",
      "800/800 [==============================] - 0s 70us/sample - loss: 3.9489 - mae: 1.6059\n",
      "Epoch 48/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 3.9307 - mae: 1.6028\n",
      "Epoch 49/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 3.9157 - mae: 1.6005\n",
      "Epoch 50/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 3.9066 - mae: 1.5978\n",
      "Epoch 51/100\n",
      "800/800 [==============================] - 0s 94us/sample - loss: 3.8976 - mae: 1.5956\n",
      "Epoch 52/100\n",
      "800/800 [==============================] - 0s 90us/sample - loss: 3.8923 - mae: 1.5958\n",
      "Epoch 53/100\n",
      "800/800 [==============================] - 0s 90us/sample - loss: 3.8874 - mae: 1.5945\n",
      "Epoch 54/100\n",
      "800/800 [==============================] - 0s 90us/sample - loss: 3.8828 - mae: 1.5933\n",
      "Epoch 55/100\n",
      "800/800 [==============================] - 0s 88us/sample - loss: 3.8795 - mae: 1.5931\n",
      "Epoch 56/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 3.8782 - mae: 1.5927\n",
      "Epoch 57/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 3.8760 - mae: 1.5924\n",
      "Epoch 58/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8748 - mae: 1.5914\n",
      "Epoch 59/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8742 - mae: 1.5913\n",
      "Epoch 60/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 3.8670 - mae: 1.5907\n",
      "Epoch 61/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 3.8741 - mae: 1.5927\n",
      "Epoch 62/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8732 - mae: 1.5919\n",
      "Epoch 63/100\n",
      "800/800 [==============================] - 0s 93us/sample - loss: 3.8724 - mae: 1.5917\n",
      "Epoch 64/100\n",
      "800/800 [==============================] - 0s 92us/sample - loss: 3.8694 - mae: 1.5909\n",
      "Epoch 65/100\n",
      "800/800 [==============================] - 0s 94us/sample - loss: 3.8717 - mae: 1.5921\n",
      "Epoch 66/100\n",
      "800/800 [==============================] - 0s 89us/sample - loss: 3.8712 - mae: 1.5910\n",
      "Epoch 67/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 3.8715 - mae: 1.5919\n",
      "Epoch 68/100\n",
      "800/800 [==============================] - 0s 80us/sample - loss: 3.8719 - mae: 1.5913\n",
      "Epoch 69/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8723 - mae: 1.5911\n",
      "Epoch 70/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8719 - mae: 1.5912\n",
      "Epoch 71/100\n",
      "800/800 [==============================] - 0s 78us/sample - loss: 3.8714 - mae: 1.5925\n",
      "Epoch 72/100\n",
      "800/800 [==============================] - 0s 111us/sample - loss: 3.8708 - mae: 1.5914\n",
      "Epoch 73/100\n",
      "800/800 [==============================] - 0s 117us/sample - loss: 3.8726 - mae: 1.5920\n",
      "Epoch 74/100\n",
      "800/800 [==============================] - 0s 87us/sample - loss: 3.8722 - mae: 1.5914\n",
      "Epoch 75/100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "800/800 [==============================] - 0s 93us/sample - loss: 3.8712 - mae: 1.5915\n",
      "Epoch 76/100\n",
      "800/800 [==============================] - 0s 88us/sample - loss: 3.8705 - mae: 1.5922\n",
      "Epoch 77/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 3.8711 - mae: 1.5916\n",
      "Epoch 78/100\n",
      "800/800 [==============================] - 0s 72us/sample - loss: 3.8703 - mae: 1.5912\n",
      "Epoch 79/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8694 - mae: 1.5907\n",
      "Epoch 80/100\n",
      "800/800 [==============================] - 0s 69us/sample - loss: 3.8707 - mae: 1.5914\n",
      "Epoch 81/100\n",
      "800/800 [==============================] - 0s 72us/sample - loss: 3.8720 - mae: 1.5923\n",
      "Epoch 82/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8718 - mae: 1.5914\n",
      "Epoch 83/100\n",
      "800/800 [==============================] - 0s 73us/sample - loss: 3.8702 - mae: 1.5917\n",
      "Epoch 84/100\n",
      "800/800 [==============================] - 0s 73us/sample - loss: 3.8710 - mae: 1.5915\n",
      "Epoch 85/100\n",
      "800/800 [==============================] - 0s 73us/sample - loss: 3.8719 - mae: 1.5922\n",
      "Epoch 86/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8713 - mae: 1.5924\n",
      "Epoch 87/100\n",
      "800/800 [==============================] - 0s 72us/sample - loss: 3.8712 - mae: 1.5910\n",
      "Epoch 88/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8704 - mae: 1.5926\n",
      "Epoch 89/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 3.8732 - mae: 1.5919\n",
      "Epoch 90/100\n",
      "800/800 [==============================] - 0s 70us/sample - loss: 3.8722 - mae: 1.5919\n",
      "Epoch 91/100\n",
      "800/800 [==============================] - 0s 76us/sample - loss: 3.8699 - mae: 1.5905\n",
      "Epoch 92/100\n",
      "800/800 [==============================] - 0s 83us/sample - loss: 3.8719 - mae: 1.5915\n",
      "Epoch 93/100\n",
      "800/800 [==============================] - 0s 73us/sample - loss: 3.8710 - mae: 1.5918\n",
      "Epoch 94/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8717 - mae: 1.5923\n",
      "Epoch 95/100\n",
      "800/800 [==============================] - 0s 75us/sample - loss: 3.8732 - mae: 1.5915\n",
      "Epoch 96/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8716 - mae: 1.5918\n",
      "Epoch 97/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8710 - mae: 1.5916\n",
      "Epoch 98/100\n",
      "800/800 [==============================] - 0s 73us/sample - loss: 3.8718 - mae: 1.5920\n",
      "Epoch 99/100\n",
      "800/800 [==============================] - 0s 74us/sample - loss: 3.8702 - mae: 1.5915\n",
      "Epoch 100/100\n",
      "800/800 [==============================] - 0s 77us/sample - loss: 3.8709 - mae: 1.5905\n",
      "w =  [[1.98907733]\n",
      " [-0.988518476]]\n",
      "b =  [3.06004095]\n",
      "WARNING:tensorflow:From /Users/alan/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "INFO:tensorflow:Assets written to: ./data/linear_model/1/assets\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow.keras import models,layers,optimizers\n",
    "\n",
    "## 样本数量\n",
    "n = 800\n",
    "\n",
    "## 生成测试用数据集\n",
    "X = tf.random.uniform([n,2],minval=-10,maxval=10) \n",
    "w0 = tf.constant([[2.0],[-1.0]])\n",
    "b0 = tf.constant(3.0)\n",
    "\n",
    "Y = X@w0 + b0 + tf.random.normal([n,1],\n",
    "    mean = 0.0,stddev= 2.0) # @表示矩阵乘法,增加正态扰动\n",
    "\n",
    "## 建立模型\n",
    "tf.keras.backend.clear_session()\n",
    "inputs = layers.Input(shape = (2,),name =\"inputs\") #设置输入名字为inputs\n",
    "outputs = layers.Dense(1, name = \"outputs\")(inputs) #设置输出名字为outputs\n",
    "linear = models.Model(inputs = inputs,outputs = outputs)\n",
    "linear.summary()\n",
    "\n",
    "## 使用fit方法进行训练\n",
    "linear.compile(optimizer=\"rmsprop\",loss=\"mse\",metrics=[\"mae\"])\n",
    "linear.fit(X,Y,batch_size = 8,epochs = 100)  \n",
    "\n",
    "tf.print(\"w = \",linear.layers[1].kernel)\n",
    "tf.print(\"b = \",linear.layers[1].bias)\n",
    "\n",
    "## 将模型保存成pb格式文件\n",
    "export_path = \"./data/linear_model/\"\n",
    "version = \"1\"       #后续可以通过版本号进行模型版本迭代与管理\n",
    "linear.save(export_path+version, save_format=\"tf\") "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1m\u001b[36massets\u001b[m\u001b[m         saved_model.pb \u001b[1m\u001b[36mvariables\u001b[m\u001b[m\r\n"
     ]
    }
   ],
   "source": [
    "#查看保存的模型文件\n",
    "!ls {export_path+version}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:\n",
      "\n",
      "signature_def['__saved_model_init_op']:\n",
      "  The given SavedModel SignatureDef contains the following input(s):\n",
      "  The given SavedModel SignatureDef contains the following output(s):\n",
      "    outputs['__saved_model_init_op'] tensor_info:\n",
      "        dtype: DT_INVALID\n",
      "        shape: unknown_rank\n",
      "        name: NoOp\n",
      "  Method name is: \n",
      "\n",
      "signature_def['serving_default']:\n",
      "  The given SavedModel SignatureDef contains the following input(s):\n",
      "    inputs['inputs'] tensor_info:\n",
      "        dtype: DT_FLOAT\n",
      "        shape: (-1, 2)\n",
      "        name: serving_default_inputs:0\n",
      "  The given SavedModel SignatureDef contains the following output(s):\n",
      "    outputs['outputs'] tensor_info:\n",
      "        dtype: DT_FLOAT\n",
      "        shape: (-1, 1)\n",
      "        name: StatefulPartitionedCall:0\n",
      "  Method name is: tensorflow/serving/predict\n",
      "WARNING:tensorflow:From /Users/alan/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "\n",
      "Defined Functions:\n",
      "  Function Name: '__call__'\n",
      "    Option #1\n",
      "      Callable with:\n",
      "        Argument #1\n",
      "          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')\n",
      "        Argument #2\n",
      "          DType: bool\n",
      "          Value: True\n",
      "        Argument #3\n",
      "          DType: NoneType\n",
      "          Value: None\n",
      "    Option #2\n",
      "      Callable with:\n",
      "        Argument #1\n",
      "          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')\n",
      "        Argument #2\n",
      "          DType: bool\n",
      "          Value: False\n",
      "        Argument #3\n",
      "          DType: NoneType\n",
      "          Value: None\n",
      "\n",
      "  Function Name: '_default_save_signature'\n",
      "    Option #1\n",
      "      Callable with:\n",
      "        Argument #1\n",
      "          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')\n",
      "\n",
      "  Function Name: 'call_and_return_all_conditional_losses'\n",
      "    Option #1\n",
      "      Callable with:\n",
      "        Argument #1\n",
      "          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')\n",
      "        Argument #2\n",
      "          DType: bool\n",
      "          Value: True\n",
      "        Argument #3\n",
      "          DType: NoneType\n",
      "          Value: None\n",
      "    Option #2\n",
      "      Callable with:\n",
      "        Argument #1\n",
      "          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name='inputs')\n",
      "        Argument #2\n",
      "          DType: bool\n",
      "          Value: False\n",
      "        Argument #3\n",
      "          DType: NoneType\n",
      "          Value: None\n"
     ]
    }
   ],
   "source": [
    "# 查看模型文件相关信息\n",
    "!saved_model_cli show --dir {export_path+str(version)} --all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 二、安装 tensorflow serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "安装 tensorflow serving 有2种主要方法：通过Docker镜像安装，通过apt安装。\n",
    "\n",
    "通过Docker镜像安装是最简单，最直接的方法，推荐采用。\n",
    "\n",
    "Docker可以理解成一种容器，其上面可以给各种不同的程序提供独立的运行环境。\n",
    "\n",
    "一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.\n",
    "\n",
    "无需算法工程师同学动手安装，以下安装过程仅供参考。\n",
    "\n",
    "不同操作系统机器上安装Docker的方法可以参照以下链接。\n",
    "\n",
    "Windows: https://www.runoob.com/docker/windows-docker-install.html\n",
    "\n",
    "MacOs: https://www.runoob.com/docker/macos-docker-install.html\n",
    "\n",
    "CentOS: https://www.runoob.com/docker/centos-docker-install.html\n",
    "\n",
    "安装Docker成功后，使用如下命令加载 tensorflow/serving 镜像到Docker中\n",
    "\n",
    "docker pull tensorflow/serving"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 三、启动 tensorflow serving 服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker run -t --rm -p 8501:8501 \\\n",
    "    -v \"/Users/.../data/linear_model/\" \\\n",
    "    -e MODEL_NAME=linear_model \\\n",
    "    tensorflow/serving & >server.log 2>&1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 四、向API服务发送请求\n",
    "\n",
    "可以使用任何编程语言的http功能发送请求，下面示范linux的 curl 命令发送请求，以及Python的requests库发送请求。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "curl: (7) Failed to connect to localhost port 8501: Connection refused\r\n"
     ]
    }
   ],
   "source": [
    "!curl -d '{\"instances\": [1.0, 2.0, 5.0]}' \\\n",
    "    -X POST http://localhost:8501/v1/models/linear_model:predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json,requests\n",
    "\n",
    "data = json.dumps({\"signature_name\": \"serving_default\", \"instances\": [[1.0, 2.0], [5.0,7.0]]})\n",
    "headers = {\"content-type\": \"application/json\"}\n",
    "json_response = requests.post('http://localhost:8501/v1/models/linear_model:predict', \n",
    "        data=data, headers=headers)\n",
    "predictions = json.loads(json_response.text)[\"predictions\"]\n",
    "print(predictions)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
